David Chiu (largitdata.com)
david@largitdata.com
2014/10/03
大數軟體(largitdata.com)有限公司創辦人
前趨勢科技工程師
ywchiu.com
立即完成統計分析
內建許多數學函式及圖形套件(也可安裝第三方套件)
免費且開源
R 是函式語言 (Functional Programming)
R 是直譯式語言 (Interpreted Language)
R 是物件導向語言 (Object Oriented Language)
最受歡迎的語言持續為 R, Python (39%), 及 SQL (37%). SAS 大約在 20%上下.
By Gregory Piatetsky, Aug 27, 2013.
R 是最廣為Kaggle比賽參與者所使用的語言
“I use R, and occasionally matlab, for data analysis. There is a large, active and extremely knowledgeable R community at Google.”
在 2007, Revolution Analytics 提供商業版本的R
Big Data Appliance, 整合R, Apache Hadoop, Oracle Enterprise Linux, 和 NoSQL 資料庫於 Exadata
data(iris)
str(iris)
'data.frame': 150 obs. of 5 variables:
$ Sepal.Length: num 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
$ Sepal.Width : num 3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
$ Petal.Length: num 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
$ Petal.Width : num 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
$ Species : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...
Five.Sepal.iris = iris[1:5, c("Sepal.Length", "Sepal.Width")]
Five.Sepal.iris
Sepal.Length Sepal.Width
1 5.1 3.5
2 4.9 3.0
3 4.7 3.2
4 4.6 3.1
5 5.0 3.6
setosa.data = iris[iris$Species=="setosa",1:5]
setosa.data
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1 5.1 3.5 1.4 0.2 setosa
2 4.9 3.0 1.4 0.2 setosa
3 4.7 3.2 1.3 0.2 setosa
4 4.6 3.1 1.5 0.2 setosa
5 5.0 3.6 1.4 0.2 setosa
6 5.4 3.9 1.7 0.4 setosa
7 4.6 3.4 1.4 0.3 setosa
8 5.0 3.4 1.5 0.2 setosa
9 4.4 2.9 1.4 0.2 setosa
10 4.9 3.1 1.5 0.1 setosa
11 5.4 3.7 1.5 0.2 setosa
12 4.8 3.4 1.6 0.2 setosa
13 4.8 3.0 1.4 0.1 setosa
14 4.3 3.0 1.1 0.1 setosa
15 5.8 4.0 1.2 0.2 setosa
16 5.7 4.4 1.5 0.4 setosa
17 5.4 3.9 1.3 0.4 setosa
18 5.1 3.5 1.4 0.3 setosa
19 5.7 3.8 1.7 0.3 setosa
20 5.1 3.8 1.5 0.3 setosa
21 5.4 3.4 1.7 0.2 setosa
22 5.1 3.7 1.5 0.4 setosa
23 4.6 3.6 1.0 0.2 setosa
24 5.1 3.3 1.7 0.5 setosa
25 4.8 3.4 1.9 0.2 setosa
26 5.0 3.0 1.6 0.2 setosa
27 5.0 3.4 1.6 0.4 setosa
28 5.2 3.5 1.5 0.2 setosa
29 5.2 3.4 1.4 0.2 setosa
30 4.7 3.2 1.6 0.2 setosa
31 4.8 3.1 1.6 0.2 setosa
32 5.4 3.4 1.5 0.4 setosa
33 5.2 4.1 1.5 0.1 setosa
34 5.5 4.2 1.4 0.2 setosa
35 4.9 3.1 1.5 0.2 setosa
36 5.0 3.2 1.2 0.2 setosa
37 5.5 3.5 1.3 0.2 setosa
38 4.9 3.6 1.4 0.1 setosa
39 4.4 3.0 1.3 0.2 setosa
40 5.1 3.4 1.5 0.2 setosa
41 5.0 3.5 1.3 0.3 setosa
42 4.5 2.3 1.3 0.3 setosa
43 4.4 3.2 1.3 0.2 setosa
44 5.0 3.5 1.6 0.6 setosa
45 5.1 3.8 1.9 0.4 setosa
46 4.8 3.0 1.4 0.3 setosa
47 5.1 3.8 1.6 0.2 setosa
48 4.6 3.2 1.4 0.2 setosa
49 5.3 3.7 1.5 0.2 setosa
50 5.0 3.3 1.4 0.2 setosa
which(iris$Species=="setosa")
[1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
[24] 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
[47] 47 48 49 50
使用subset 取欄位
sepal.data = subset(iris, select=c("Sepal.Length", "Sepal.Width"))
sepal.data
Sepal.Length Sepal.Width
1 5.1 3.5
2 4.9 3.0
3 4.7 3.2
4 4.6 3.1
5 5.0 3.6
6 5.4 3.9
7 4.6 3.4
8 5.0 3.4
9 4.4 2.9
10 4.9 3.1
11 5.4 3.7
12 4.8 3.4
13 4.8 3.0
14 4.3 3.0
15 5.8 4.0
16 5.7 4.4
17 5.4 3.9
18 5.1 3.5
19 5.7 3.8
20 5.1 3.8
21 5.4 3.4
22 5.1 3.7
23 4.6 3.6
24 5.1 3.3
25 4.8 3.4
26 5.0 3.0
27 5.0 3.4
28 5.2 3.5
29 5.2 3.4
30 4.7 3.2
31 4.8 3.1
32 5.4 3.4
33 5.2 4.1
34 5.5 4.2
35 4.9 3.1
36 5.0 3.2
37 5.5 3.5
38 4.9 3.6
39 4.4 3.0
40 5.1 3.4
41 5.0 3.5
42 4.5 2.3
43 4.4 3.2
44 5.0 3.5
45 5.1 3.8
46 4.8 3.0
47 5.1 3.8
48 4.6 3.2
49 5.3 3.7
50 5.0 3.3
51 7.0 3.2
52 6.4 3.2
53 6.9 3.1
54 5.5 2.3
55 6.5 2.8
56 5.7 2.8
57 6.3 3.3
58 4.9 2.4
59 6.6 2.9
60 5.2 2.7
61 5.0 2.0
62 5.9 3.0
63 6.0 2.2
64 6.1 2.9
65 5.6 2.9
66 6.7 3.1
67 5.6 3.0
68 5.8 2.7
69 6.2 2.2
70 5.6 2.5
71 5.9 3.2
72 6.1 2.8
73 6.3 2.5
74 6.1 2.8
75 6.4 2.9
76 6.6 3.0
77 6.8 2.8
78 6.7 3.0
79 6.0 2.9
80 5.7 2.6
81 5.5 2.4
82 5.5 2.4
83 5.8 2.7
84 6.0 2.7
85 5.4 3.0
86 6.0 3.4
87 6.7 3.1
88 6.3 2.3
89 5.6 3.0
90 5.5 2.5
91 5.5 2.6
92 6.1 3.0
93 5.8 2.6
94 5.0 2.3
95 5.6 2.7
96 5.7 3.0
97 5.7 2.9
98 6.2 2.9
99 5.1 2.5
100 5.7 2.8
101 6.3 3.3
102 5.8 2.7
103 7.1 3.0
104 6.3 2.9
105 6.5 3.0
106 7.6 3.0
107 4.9 2.5
108 7.3 2.9
109 6.7 2.5
110 7.2 3.6
111 6.5 3.2
112 6.4 2.7
113 6.8 3.0
114 5.7 2.5
115 5.8 2.8
116 6.4 3.2
117 6.5 3.0
118 7.7 3.8
119 7.7 2.6
120 6.0 2.2
121 6.9 3.2
122 5.6 2.8
123 7.7 2.8
124 6.3 2.7
125 6.7 3.3
126 7.2 3.2
127 6.2 2.8
128 6.1 3.0
129 6.4 2.8
130 7.2 3.0
131 7.4 2.8
132 7.9 3.8
133 6.4 2.8
134 6.3 2.8
135 6.1 2.6
136 7.7 3.0
137 6.3 3.4
138 6.4 3.1
139 6.0 3.0
140 6.9 3.1
141 6.7 3.1
142 6.9 3.1
143 5.8 2.7
144 6.8 3.2
145 6.7 3.3
146 6.7 3.0
147 6.3 2.5
148 6.5 3.0
149 6.2 3.4
150 5.9 3.0
以Species 做篩選條件
setosa.data = subset(iris, Species =="setosa")
setosa.data
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1 5.1 3.5 1.4 0.2 setosa
2 4.9 3.0 1.4 0.2 setosa
3 4.7 3.2 1.3 0.2 setosa
4 4.6 3.1 1.5 0.2 setosa
5 5.0 3.6 1.4 0.2 setosa
6 5.4 3.9 1.7 0.4 setosa
7 4.6 3.4 1.4 0.3 setosa
8 5.0 3.4 1.5 0.2 setosa
9 4.4 2.9 1.4 0.2 setosa
10 4.9 3.1 1.5 0.1 setosa
11 5.4 3.7 1.5 0.2 setosa
12 4.8 3.4 1.6 0.2 setosa
13 4.8 3.0 1.4 0.1 setosa
14 4.3 3.0 1.1 0.1 setosa
15 5.8 4.0 1.2 0.2 setosa
16 5.7 4.4 1.5 0.4 setosa
17 5.4 3.9 1.3 0.4 setosa
18 5.1 3.5 1.4 0.3 setosa
19 5.7 3.8 1.7 0.3 setosa
20 5.1 3.8 1.5 0.3 setosa
21 5.4 3.4 1.7 0.2 setosa
22 5.1 3.7 1.5 0.4 setosa
23 4.6 3.6 1.0 0.2 setosa
24 5.1 3.3 1.7 0.5 setosa
25 4.8 3.4 1.9 0.2 setosa
26 5.0 3.0 1.6 0.2 setosa
27 5.0 3.4 1.6 0.4 setosa
28 5.2 3.5 1.5 0.2 setosa
29 5.2 3.4 1.4 0.2 setosa
30 4.7 3.2 1.6 0.2 setosa
31 4.8 3.1 1.6 0.2 setosa
32 5.4 3.4 1.5 0.4 setosa
33 5.2 4.1 1.5 0.1 setosa
34 5.5 4.2 1.4 0.2 setosa
35 4.9 3.1 1.5 0.2 setosa
36 5.0 3.2 1.2 0.2 setosa
37 5.5 3.5 1.3 0.2 setosa
38 4.9 3.6 1.4 0.1 setosa
39 4.4 3.0 1.3 0.2 setosa
40 5.1 3.4 1.5 0.2 setosa
41 5.0 3.5 1.3 0.3 setosa
42 4.5 2.3 1.3 0.3 setosa
43 4.4 3.2 1.3 0.2 setosa
44 5.0 3.5 1.6 0.6 setosa
45 5.1 3.8 1.9 0.4 setosa
46 4.8 3.0 1.4 0.3 setosa
47 5.1 3.8 1.6 0.2 setosa
48 4.6 3.2 1.4 0.2 setosa
49 5.3 3.7 1.5 0.2 setosa
50 5.0 3.3 1.4 0.2 setosa
可以做篩選條件的組合
example.data= subset(iris, Petal.Length <=1.4 & Petal.Width >= 0.2, select=Species )
example.data
Species
1 setosa
2 setosa
3 setosa
5 setosa
7 setosa
9 setosa
15 setosa
17 setosa
18 setosa
23 setosa
29 setosa
34 setosa
36 setosa
37 setosa
39 setosa
41 setosa
42 setosa
43 setosa
46 setosa
48 setosa
50 setosa
以Merge 做資料合併
flower.type = data.frame(Species = "setosa", Flower = "iris")
merge(flower.type, iris[1:3,], by ="Species")
Species Flower Sepal.Length Sepal.Width Petal.Length Petal.Width
1 setosa iris 5.1 3.5 1.4 0.2
2 setosa iris 4.9 3.0 1.4 0.2
3 setosa iris 4.7 3.2 1.3 0.2
用order做資料排序
iris[order(iris$Sepal.Length, decreasing = TRUE),]
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
132 7.9 3.8 6.4 2.0 virginica
118 7.7 3.8 6.7 2.2 virginica
119 7.7 2.6 6.9 2.3 virginica
123 7.7 2.8 6.7 2.0 virginica
136 7.7 3.0 6.1 2.3 virginica
106 7.6 3.0 6.6 2.1 virginica
131 7.4 2.8 6.1 1.9 virginica
108 7.3 2.9 6.3 1.8 virginica
110 7.2 3.6 6.1 2.5 virginica
126 7.2 3.2 6.0 1.8 virginica
130 7.2 3.0 5.8 1.6 virginica
103 7.1 3.0 5.9 2.1 virginica
51 7.0 3.2 4.7 1.4 versicolor
53 6.9 3.1 4.9 1.5 versicolor
121 6.9 3.2 5.7 2.3 virginica
140 6.9 3.1 5.4 2.1 virginica
142 6.9 3.1 5.1 2.3 virginica
77 6.8 2.8 4.8 1.4 versicolor
113 6.8 3.0 5.5 2.1 virginica
144 6.8 3.2 5.9 2.3 virginica
66 6.7 3.1 4.4 1.4 versicolor
78 6.7 3.0 5.0 1.7 versicolor
87 6.7 3.1 4.7 1.5 versicolor
109 6.7 2.5 5.8 1.8 virginica
125 6.7 3.3 5.7 2.1 virginica
141 6.7 3.1 5.6 2.4 virginica
145 6.7 3.3 5.7 2.5 virginica
146 6.7 3.0 5.2 2.3 virginica
59 6.6 2.9 4.6 1.3 versicolor
76 6.6 3.0 4.4 1.4 versicolor
55 6.5 2.8 4.6 1.5 versicolor
105 6.5 3.0 5.8 2.2 virginica
111 6.5 3.2 5.1 2.0 virginica
117 6.5 3.0 5.5 1.8 virginica
148 6.5 3.0 5.2 2.0 virginica
52 6.4 3.2 4.5 1.5 versicolor
75 6.4 2.9 4.3 1.3 versicolor
112 6.4 2.7 5.3 1.9 virginica
116 6.4 3.2 5.3 2.3 virginica
129 6.4 2.8 5.6 2.1 virginica
133 6.4 2.8 5.6 2.2 virginica
138 6.4 3.1 5.5 1.8 virginica
57 6.3 3.3 4.7 1.6 versicolor
73 6.3 2.5 4.9 1.5 versicolor
88 6.3 2.3 4.4 1.3 versicolor
101 6.3 3.3 6.0 2.5 virginica
104 6.3 2.9 5.6 1.8 virginica
124 6.3 2.7 4.9 1.8 virginica
134 6.3 2.8 5.1 1.5 virginica
137 6.3 3.4 5.6 2.4 virginica
147 6.3 2.5 5.0 1.9 virginica
69 6.2 2.2 4.5 1.5 versicolor
98 6.2 2.9 4.3 1.3 versicolor
127 6.2 2.8 4.8 1.8 virginica
149 6.2 3.4 5.4 2.3 virginica
64 6.1 2.9 4.7 1.4 versicolor
72 6.1 2.8 4.0 1.3 versicolor
74 6.1 2.8 4.7 1.2 versicolor
92 6.1 3.0 4.6 1.4 versicolor
128 6.1 3.0 4.9 1.8 virginica
135 6.1 2.6 5.6 1.4 virginica
63 6.0 2.2 4.0 1.0 versicolor
79 6.0 2.9 4.5 1.5 versicolor
84 6.0 2.7 5.1 1.6 versicolor
86 6.0 3.4 4.5 1.6 versicolor
120 6.0 2.2 5.0 1.5 virginica
139 6.0 3.0 4.8 1.8 virginica
62 5.9 3.0 4.2 1.5 versicolor
71 5.9 3.2 4.8 1.8 versicolor
150 5.9 3.0 5.1 1.8 virginica
15 5.8 4.0 1.2 0.2 setosa
68 5.8 2.7 4.1 1.0 versicolor
83 5.8 2.7 3.9 1.2 versicolor
93 5.8 2.6 4.0 1.2 versicolor
102 5.8 2.7 5.1 1.9 virginica
115 5.8 2.8 5.1 2.4 virginica
143 5.8 2.7 5.1 1.9 virginica
16 5.7 4.4 1.5 0.4 setosa
19 5.7 3.8 1.7 0.3 setosa
56 5.7 2.8 4.5 1.3 versicolor
80 5.7 2.6 3.5 1.0 versicolor
96 5.7 3.0 4.2 1.2 versicolor
97 5.7 2.9 4.2 1.3 versicolor
100 5.7 2.8 4.1 1.3 versicolor
114 5.7 2.5 5.0 2.0 virginica
65 5.6 2.9 3.6 1.3 versicolor
67 5.6 3.0 4.5 1.5 versicolor
70 5.6 2.5 3.9 1.1 versicolor
89 5.6 3.0 4.1 1.3 versicolor
95 5.6 2.7 4.2 1.3 versicolor
122 5.6 2.8 4.9 2.0 virginica
34 5.5 4.2 1.4 0.2 setosa
37 5.5 3.5 1.3 0.2 setosa
54 5.5 2.3 4.0 1.3 versicolor
81 5.5 2.4 3.8 1.1 versicolor
82 5.5 2.4 3.7 1.0 versicolor
90 5.5 2.5 4.0 1.3 versicolor
91 5.5 2.6 4.4 1.2 versicolor
6 5.4 3.9 1.7 0.4 setosa
11 5.4 3.7 1.5 0.2 setosa
17 5.4 3.9 1.3 0.4 setosa
21 5.4 3.4 1.7 0.2 setosa
32 5.4 3.4 1.5 0.4 setosa
85 5.4 3.0 4.5 1.5 versicolor
49 5.3 3.7 1.5 0.2 setosa
28 5.2 3.5 1.5 0.2 setosa
29 5.2 3.4 1.4 0.2 setosa
33 5.2 4.1 1.5 0.1 setosa
60 5.2 2.7 3.9 1.4 versicolor
1 5.1 3.5 1.4 0.2 setosa
18 5.1 3.5 1.4 0.3 setosa
20 5.1 3.8 1.5 0.3 setosa
22 5.1 3.7 1.5 0.4 setosa
24 5.1 3.3 1.7 0.5 setosa
40 5.1 3.4 1.5 0.2 setosa
45 5.1 3.8 1.9 0.4 setosa
47 5.1 3.8 1.6 0.2 setosa
99 5.1 2.5 3.0 1.1 versicolor
5 5.0 3.6 1.4 0.2 setosa
8 5.0 3.4 1.5 0.2 setosa
26 5.0 3.0 1.6 0.2 setosa
27 5.0 3.4 1.6 0.4 setosa
36 5.0 3.2 1.2 0.2 setosa
41 5.0 3.5 1.3 0.3 setosa
44 5.0 3.5 1.6 0.6 setosa
50 5.0 3.3 1.4 0.2 setosa
61 5.0 2.0 3.5 1.0 versicolor
94 5.0 2.3 3.3 1.0 versicolor
2 4.9 3.0 1.4 0.2 setosa
10 4.9 3.1 1.5 0.1 setosa
35 4.9 3.1 1.5 0.2 setosa
38 4.9 3.6 1.4 0.1 setosa
58 4.9 2.4 3.3 1.0 versicolor
107 4.9 2.5 4.5 1.7 virginica
12 4.8 3.4 1.6 0.2 setosa
13 4.8 3.0 1.4 0.1 setosa
25 4.8 3.4 1.9 0.2 setosa
31 4.8 3.1 1.6 0.2 setosa
46 4.8 3.0 1.4 0.3 setosa
3 4.7 3.2 1.3 0.2 setosa
30 4.7 3.2 1.6 0.2 setosa
4 4.6 3.1 1.5 0.2 setosa
7 4.6 3.4 1.4 0.3 setosa
23 4.6 3.6 1.0 0.2 setosa
48 4.6 3.2 1.4 0.2 setosa
42 4.5 2.3 1.3 0.3 setosa
9 4.4 2.9 1.4 0.2 setosa
39 4.4 3.0 1.3 0.2 setosa
43 4.4 3.2 1.3 0.2 setosa
14 4.3 3.0 1.1 0.1 setosa
可以做單一變數統計
x = c(1,2,3,4,5,6,7,8,9,10)
mean(x)
[1] 5.5
min(x)
[1] 1
median(x)
[1] 5.5
可以做單一變數統計
max(x)
[1] 10
var(x)
[1] 9.167
summary(x)
Min. 1st Qu. Median Mean 3rd Qu. Max.
1.00 3.25 5.50 5.50 7.75 10.00
透過sapply針對多個欄位操作
sapply(iris[1:4], mean, na.rm=TRUE)
Sepal.Length Sepal.Width Petal.Length Petal.Width
5.843 3.057 3.758 1.199
可用來分割、套用及合併資料的函式
library(plyr)
ddply(iris, c("Species"), function(df) mean(df$Sepal.Length))
Species V1
1 setosa 5.006
2 versicolor 5.936
3 virginica 6.588
Pie Chart
table.iris = table(iris$Species)
pie(table.iris)
Histogram
hist(iris$Sepal.Length)
Box Plot
boxplot(Petal.Width ~ Species, data = iris)
Scatter Plot
plot(x=iris$Petal.Length, y=iris$Petal.Width, col=iris$Species)
Scatter Plot
suppressPackageStartupMessages(library(ggplot2))
library(ggplot2)
qplot(Sepal.Length, Petal.Length, data = iris, color = Species)
suppressPackageStartupMessages(library(googleVis))
library(googleVis)
GeoMarker <- gvisGeoChart(Andrew, "LatLong", sizevar='Speed_kt',colorvar="Pressure_mb", options=list(region="US"))
plot(GeoMarker)
A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E
– Tom Mitchell (1998)
Given number of friends x, predict how many goods I will receive on each Facebook posts
dataset <- read.csv('fbgood.txt',head=TRUE, sep='\t', row.names=1)
x = dataset$friends
y = dataset$getgoods
plot(x,y)
plot(x,y)
fit <- lm(y ~ x)
abline(fit, col = 'red', lwd=3)
plot(x,y)
polyfit2 <- lm(y ~ poly(x, 2))
lines(sort(x), polyfit2$fit[order(x)], col = 2, lwd = 3)
plot(x,y)
polyfit3 <- lm(y ~ poly(x, 3));
lines(sort(x), polyfit3$fit[order(x)], col = 2, lwd = 3)
Given features of bank costumer, predict whether the client will subscribe a term deposit
Features:
Labels:
library(e1071)
dataset <- read.csv('bank.csv',head=TRUE, sep=';')
set.seed(2)
ind <- sample(2, nrow(dataset), replace = TRUE, prob=c(0.7, 0.3))
train = dataset[ind == 1,]
test = dataset[ind == 2,]
model <- svm(y~., data = train, probability = TRUE)
pred <- predict(model, test[,1:(dim(test)[[2]]-1)], probability = TRUE)
table(pred,test[,dim(test)[2]])
pred no yes
no 1196 98
yes 25 46
library(caret)
confusionMatrix(table(pred,test[,dim(test)[2]]))
Confusion Matrix and Statistics
pred no yes
no 1196 98
yes 25 46
Accuracy : 0.91
95% CI : (0.893, 0.925)
No Information Rate : 0.895
P-Value [Acc > NIR] : 0.0333
Kappa : 0.385
Mcnemar's Test P-Value : 8.47e-11
Sensitivity : 0.980
Specificity : 0.319
Pos Pred Value : 0.924
Neg Pred Value : 0.648
Prevalence : 0.895
Detection Rate : 0.876
Detection Prevalence : 0.948
Balanced Accuracy : 0.649
'Positive' Class : no
library(ROCR)
pred.prob <- attr(pred, "probabilities")
pred.to.roc <- pred.prob[, 2]
pred.rocr <- prediction(pred.to.roc, as.factor(test[,(dim(test)[[2]])]))
perf.rocr <- performance(pred.rocr, measure = "auc", x.measure = "cutoff")
perf.tpr.rocr <- performance(pred.rocr, "tpr","fpr")
plot(perf.tpr.rocr, colorize=T,main=paste("AUC:",(perf.rocr@y.values)))
Calculate a new index to measure economy index of each Taiwan city/county
–2012年《天下雜誌》幸福城市大調查 - 第505期
dataset <- read.csv('eco_index.csv',head=TRUE, sep=',', row.names=1)
pc.cr <- princomp(dataset, cor = TRUE)
plot(pc.cr)
screeplot(pc.cr, type="lines")
abline(h=1, lty=3)
biplot(pc.cr)
barplot(sort(-pc.cr$scores[,1], TRUE))
Segment customers based on existing features
Clustering by 4 features
mydata <- read.csv('costumer_segment.txt',head=TRUE, sep='\t')
mydata <- scale(mydata)
d <- dist(mydata, method = "euclidean")
fit <- hclust(d, method="ward")
plot(fit)
plot(fit)
k1 = 4
groups <- cutree(fit, k=k1)
rect.hclust(fit, k=k1, border="red")
fit <- kmeans(mydata, k1)
plot(mydata, col = fit$cluster)
library(cluster)
clusplot(mydata, fit$cluster, color=TRUE, shade=TRUE, lines=0)
可以讓開發者以其他語言撰寫Mapper/Reducer(R, python, perl)
Mapper, reducer, 及 combiner 程序可以用來導向I/O
Streaming 會因為要開啟腳本語言的虛擬機器,因此會有額外的時間與空間負擔
Author: David Chiu
Contact: david@largitdata.com